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Introduction 

The  main  thrust  of  the  research  conducted  during  this  period  has  been 
toward  the  development  of  efficient  algorithms  for  solving  the  finite  - 
dimensional  constrained  optimization  problem 
minimize  f(x) 

subject  to  g(x)  <  0  and  h(x)  =  0. 

where  f:Hn  •+  F  ,  g : H  n  -*  and  h:Hn  ■>  -P  p  are  smooth  functions. 

Historically,  problems  of  this  type  have  been  solved  by  either  penalty 
function  methods  or  through  linearization  procedures.  The  fact  that  neither 
of  these  techniques  is  completely  satisfactory  for  general  nonlinear  problems 
has  led  to  a  concentrated  research  effort  to  find  better  approaches.  What 
has  so  far  emerged  from  this  work  is  a  blending  of  the  penalty  function  and 
linearization  ideas  with  the  quadratic  approximation  methods  associated  with 
unconstrained  optimization.  While  there  remain  many  unresolved  issues, 
it  is  now  apparent  that  this  synthesis  has  resulted  in  more  efficient 
algorithms  for  the  nonlinear  constrained  optimization  problem.  The  research 
described  briefly  below  has  been  part  of  this  development. 

Penalty  functions 

A  family  of  multiplier  functions,  M(x,x;c,d),  has  been  developed  for 
the  equality-constrained  problem.  M  depends  upon  two  parameters  (c,d)  and 
is  concave  and  quadratic  in  the  multiplier  A.  For  certain  finite  values 
of  c  and  d  the  pair  (x*,A*)  is  a  saddle  point  of  M  if  and  only  if  (x*,a*) 
is  an  optimal  vector  and  its  multiplier  for  the  problem.  If  v  M(x,x;c,d)  =  0 
is  solved  for  A  in  terms  of  x,  then  for  appropriate  values  of  c  and  d, 
M(x,x(x);  c,d)  is  an  exact  penalty  function  for  the  optimization  problem. 

The  penalty  function  and  its  applications  are  discussed  in  [IT],  [2T],  [3T], 
[6T],  [IP],  and  [2P]. 
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Constrained  optimization  and  differential  equations. 

It  is  well  known  that  for  the  unconstrained  minimization,  solving  the 
gradient  differential  equation,  x'  =  -vf(x),  x(0)  =  x°,  by  Euler's  method 
is  equivalent  to  using  a  steepest  descent  algorithm  to  minimize  f.  In  [5T], 
[7T]  and  [IP]  an  algorithm  for  solving  the  equality-constrained  problem 
is  developed  which  is  based  on  the  gradient  system 

x'  =  -vxM(x,A;c,d)  ,  x(0)  =  x°  , 

A'  =  -vxM(x,A;c,d)  ,  a(0)  =  A0  . 

where  M  is  the  multiplier  function  reported  above.  Because  of  the  large  values 
of  the  parameter  c  which  are  required  to  validate  the  important  theoretical 
properties  of  M,  the  above  system  is  best  analyzed  as  a  singularly  perturbed 
system  of  the  form 

^  x'  =  <|i(x,A;c,d) 

A'  =  <l»(x,A;c,d)  . 

Quasi  Newton  methods. 

An  important  new  technique  for  solving  the  constrained  nonlinear  problem 
is  the  method  of  successive  quadratic  programming.  Locally  this  method  is 
equivalent  to  solving  the  nonlinear  system 
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v/(x,x)  -  0  , 
h(x)  *  0  , 

9-j (x)  =  0  ,  id 

where  -£(x,x)  is  the  Lagrangian  function  and  I  is  the  index  set  of  active 
inequality  constraints  at  optimality.  In  a  series  of  reports  and  papers, 

[4T],  [8T],  [3P],  and  [4P]  the  local  convergence  of  quasi-Newton  algorithms 
for  solving  this  system  is  investigated.  In  particular,  necessary  conditions 
for  convergence  and  necessary  and  sufficient  conditions  for  superlinear 
convergence  are  obtained. 

Implementation 

Research  continues  on  the  implementation  of  an  efficient  algorithm 
which  is  both  robust  and  locally  superli nearly  convergent.  One  major 
difficulty  which  has  been  overcome  in  this  quest  is  the  identification 
of  a  merit  function  (a  function  used  to  measure  the  progress  of  the  algorithm) 
which  is  consistent  with  superlinear  convergence.  In  [9T],  [1QT],  [5P],  and 
[6P]  a  member  of  the  family  of  multiplier  functions  described  above  is 
shown  to  have  this  desirable  property.  This  merit  function  is  incorporated 
into  an  algorithmic  scheme  for  solving  the  nonlinear  problem. 

Continuing  investigations. 

The  research  initiated  under  the  sponsorship  of  the  Army  Research  Office 
is  continuing.  Current  topics  of  interest  include  the  application  of  successive 
quadratic  programming  to  large  scale  nonlinear  problems  [11T]  and  the  implemen¬ 
tation  of  the  algorithm  described  in  [6P]  in  a  more  efficient  and  robust 
manner  [12T]. 
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